<!DOCTYPE html>
<html lang="zh">
<head><meta name="generator" content="Hexo 3.8.0">
    <meta charset="utf-8">
<title>Opentracing &amp; Jaeger - 小琦的博客</title>
<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1">



    <meta name="description" content="背景大型互联网公司为什么需要分布式跟踪系统？为了支撑日益增长的庞大业务量，我们会把服务进行整合、拆分，使我们的服务不仅能通过集群部署抵挡流量的冲击，又能根据业务在其上进行灵活的扩展。一次请求少则经过三四次服务调用完成，多则跨越几十个甚至是上百个服务节点。如何动态展示服务的链路？如何分析服务链路的瓶颈并对其进行调优？如何快速进行服务链路的故障发现？这就是服务跟踪系统存在的目的和意义。">
<meta name="keywords" content="Opentracing,Jaeger,分布式追踪">
<meta property="og:type" content="article">
<meta property="og:title" content="Opentracing &amp; Jaeger">
<meta property="og:url" content="https://vincentqi.gitee.io/blog/2019/06/02/Opentracing-Jaeger/index.html">
<meta property="og:site_name" content="小琦的博客">
<meta property="og:description" content="背景大型互联网公司为什么需要分布式跟踪系统？为了支撑日益增长的庞大业务量，我们会把服务进行整合、拆分，使我们的服务不仅能通过集群部署抵挡流量的冲击，又能根据业务在其上进行灵活的扩展。一次请求少则经过三四次服务调用完成，多则跨越几十个甚至是上百个服务节点。如何动态展示服务的链路？如何分析服务链路的瓶颈并对其进行调优？如何快速进行服务链路的故障发现？这就是服务跟踪系统存在的目的和意义。">
<meta property="og:locale" content="zh-CN">
<meta property="og:image" content="https://vincentqi.gitee.io/blog/blog/gallery/thumbnails/opentracing.png">
<meta property="og:updated_time" content="2019-06-02T09:46:58.323Z">
<meta name="twitter:card" content="summary">
<meta name="twitter:title" content="Opentracing &amp; Jaeger">
<meta name="twitter:description" content="背景大型互联网公司为什么需要分布式跟踪系统？为了支撑日益增长的庞大业务量，我们会把服务进行整合、拆分，使我们的服务不仅能通过集群部署抵挡流量的冲击，又能根据业务在其上进行灵活的扩展。一次请求少则经过三四次服务调用完成，多则跨越几十个甚至是上百个服务节点。如何动态展示服务的链路？如何分析服务链路的瓶颈并对其进行调优？如何快速进行服务链路的故障发现？这就是服务跟踪系统存在的目的和意义。">
<meta name="twitter:image" content="https://vincentqi.gitee.io/blog/blog/gallery/thumbnails/opentracing.png">







<link rel="icon" href="/blog/images/favicon.svg">


<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/bulma@0.7.2/css/bulma.css">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css">
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Ubuntu:400,600|Source+Code+Pro">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/highlight.js@9.12.0/styles/atom-one-light.css">


    
    
    
    <style>body>.footer,body>.navbar,body>.section{opacity:0}</style>
    

    
    
    
    <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/lightgallery@1.6.8/dist/css/lightgallery.min.css">
    <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/justifiedGallery@3.7.0/dist/css/justifiedGallery.min.css">
    

    
    

<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/outdatedbrowser@1.1.5/outdatedbrowser/outdatedbrowser.min.css">


    
    
    
    

<link rel="stylesheet" href="/blog/css/back-to-top.css">


    
    

    
    
    
    

    
    
<link rel="stylesheet" href="/blog/css/progressbar.css">
<script src="https://cdn.jsdelivr.net/npm/pace-js@1.0.2/pace.min.js"></script>

    
    
    

    
    
    


<link rel="stylesheet" href="/blog/css/style.css">
</head>
<body class="is-2-column">
    <nav class="navbar navbar-main">
    <div class="container">
        <div class="navbar-brand is-flex-center">
            <a class="navbar-item navbar-logo" href="/blog/">
            
                <img src="/blog/images/logo.png" alt="Opentracing &amp; Jaeger" height="28">
            
            </a>
        </div>
        <div class="navbar-menu">
            
            <div class="navbar-start">
                
                <a class="navbar-item" href="/blog/">全部</a>
                
                <a class="navbar-item" href="/blog/archives">归档</a>
                
                <a class="navbar-item" href="/blog/categories">分类</a>
                
                <a class="navbar-item" href="/blog/tags">标签</a>
                
                <a class="navbar-item" href="/blog/about">关于我</a>
                
            </div>
            
            <div class="navbar-end">
                
                    
                    
                    <a class="navbar-item" target="_blank" title="Download on GitHub" href="https://github.com/ppoffice/hexo-theme-icarus">
                        
                        <i class="fab fa-github"></i>
                        
                    </a>
                    
                
                
                
                <a class="navbar-item search" title="搜索" href="javascript:;">
                    <i class="fas fa-search"></i>
                </a>
                
            </div>
        </div>
    </div>
</nav>
    
    <section class="section">
        <div class="container">
            <div class="columns">
                <div class="column is-8-tablet is-8-desktop is-9-widescreen has-order-2 column-main"><div class="card">
    
    <div class="card-image">
        <span class="image is-7by1">
            <img class="thumbnail" src="/blog/gallery/thumbnails/opentracing.png" alt="Opentracing &amp; Jaeger">
        </span>
    </div>
    
    <div class="card-content article ">
        
        <div class="level article-meta is-size-7 is-uppercase is-mobile is-overflow-x-auto">
            <div class="level-left">
                <time class="level-item has-text-grey" datetime="2019-06-02T09:10:52.000Z">2019-06-02</time>
                
                <div class="level-item">
                <a class="has-link-grey -link" href="/blog/categories/APM/">APM</a>&nbsp;/&nbsp;<a class="has-link-grey -link" href="/blog/categories/APM/Opentracing/">Opentracing</a>
                </div>
                
                
                <span class="level-item has-text-grey">
                    
                    
                    12 分钟 读完 (大约 1750 个字)
                </span>
                
                
            </div>
        </div>
        
        <h1 class="title is-size-3 is-size-4-mobile has-text-weight-normal">
            
                Opentracing &amp; Jaeger
            
        </h1>
        <div class="content">
            <h1 id="背景"><a href="#背景" class="headerlink" title="背景"></a>背景</h1><p>大型互联网公司为什么需要分布式跟踪系统？为了支撑日益增长的庞大业务量，我们会把服务进行整合、拆分，使我们的服务不仅能通过集群部署抵挡流量的冲击，又能根据业务在其上进行灵活的扩展。一次请求少则经过三四次服务调用完成，多则跨越几十个甚至是上百个服务节点。如何动态展示服务的链路？如何分析服务链路的瓶颈并对其进行调优？如何快速进行服务链路的故障发现？这就是服务跟踪系统存在的目的和意义。<br> <a id="more"></a></p>
<p><img src="http://ata2-img.cn-hangzhou.img-pub.aliyun-inc.com/9e95e41a2fa724dc66b2f49a967845c6.png" alt="架构演进"></p>
<p>应用架构开始从单体系统逐步转变为微服务，其中的业务逻辑随之而来就会变成微服务之间的调用与请求。资源角度来看，传统服务器这个物理单位也逐渐淡化，变成了看不见摸不到的虚拟资源模式。这种弹性、标准化的架构背后，原先运维与诊断的需求也变得越来越复杂。为了应对这种变化趋势，诞生一系列面向 DevOps 的诊断与分析系统，包括集中式日志系统（Logging），集中式度量系统（Metrics）和分布式追踪系统（Tracing）。</p>
<h3 id="Logging，Metrics-和-Tracing"><a href="#Logging，Metrics-和-Tracing" class="headerlink" title="Logging，Metrics 和 Tracing"></a>Logging，Metrics 和 Tracing</h3><p>Logging，Metrics 和 Tracing 有各自专注的部分。</p>
<ul>
<li>Logging - 用于记录离散的事件。例如，应用程序的调试信息或错误信息。它是我们诊断问题的依据。</li>
<li>Metrics - 用于记录可聚合的数据。例如，队列的当前深度可被定义为一个度量值，在元素入队或出队时被更新；HTTP 请求个数可被定义为一个计数器，新请求到来时进行累加。</li>
<li>Tracing - 用于记录请求范围内的信息。例如，一次远程方法调用的执行过程和耗时。它是我们排查系统性能问题的利器。</li>
</ul>
<p><img src="http://ata2-img.cn-hangzhou.img-pub.aliyun-inc.com/92806aa2426813a4f47e6ba9b01f76f7.png" alt="image"></p>
<h3 id="常见的分布式跟踪系统"><a href="#常见的分布式跟踪系统" class="headerlink" title="常见的分布式跟踪系统"></a>常见的分布式跟踪系统</h3><ul>
<li>Spring Cloud Zipkin</li>
<li>SkyWalking</li>
<li>Jaeger</li>
</ul>
<h2 id="Jaeger是什么"><a href="#Jaeger是什么" class="headerlink" title="Jaeger是什么"></a>Jaeger是什么</h2><p>Jaeger是Uber开发的一套分布式追踪系统，已在Uber大规模使用。并在2017-9-13 加入CNCF 开源组织。使用Jaeger可以非常直观的展示整个分布式系统的调用链，由此可以很好发现和解决问题</p>
<h3 id="Jaeger架构图"><a href="#Jaeger架构图" class="headerlink" title="Jaeger架构图"></a>Jaeger架构图</h3><p><img src="http://ata2-img.cn-hangzhou.img-pub.aliyun-inc.com/f671cb89ab5946b181443c53962cfa2f.png" alt="Jaeger架构图"></p>
<h3 id="自带DEMO的效果展示"><a href="#自带DEMO的效果展示" class="headerlink" title="自带DEMO的效果展示"></a>自带DEMO的效果展示</h3><p><img src="https://raw.githubusercontent.com/jaegertracing/jaeger-ui/master/media/ss_search.png" alt="UI"></p>
<h2 id="接入方式"><a href="#接入方式" class="headerlink" title="接入方式"></a>接入方式</h2><h3 id="接入步骤"><a href="#接入步骤" class="headerlink" title="接入步骤"></a>接入步骤</h3><ol>
<li>在maven中添加web监控依赖</li>
</ol>
<figure class="highlight xml hljs"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="hljs-tag">&lt;<span class="hljs-name">dependency</span>&gt;</span></span><br><span class="line">	<span class="hljs-tag">&lt;<span class="hljs-name">groupId</span>&gt;</span>io.opentracing.contrib<span class="hljs-tag">&lt;/<span class="hljs-name">groupId</span>&gt;</span></span><br><span class="line">	<span class="hljs-tag">&lt;<span class="hljs-name">artifactId</span>&gt;</span>opentracing-spring-jaeger-web-starter<span class="hljs-tag">&lt;/<span class="hljs-name">artifactId</span>&gt;</span></span><br><span class="line">	<span class="hljs-tag">&lt;<span class="hljs-name">version</span>&gt;</span>0.2.2-SNAPSHOT<span class="hljs-tag">&lt;/<span class="hljs-name">version</span>&gt;</span></span><br><span class="line"><span class="hljs-tag">&lt;/<span class="hljs-name">dependency</span>&gt;</span></span><br></pre></td></tr></table></figure>

<p>在项目的application.yml中添加监控开启的配置信息</p>
<figure class="highlight yml hljs"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line"><span class="hljs-attr">spring:</span></span><br><span class="line"><span class="hljs-attr">  application:</span></span><br><span class="line"><span class="hljs-attr">    name:</span> <span class="hljs-string">xxx</span></span><br><span class="line"><span class="hljs-attr">    namespace:</span> <span class="hljs-string">xxx</span> <span class="hljs-comment">#配置项目的命名空间</span></span><br><span class="line"><span class="hljs-attr">opentracing:</span></span><br><span class="line"><span class="hljs-attr">  jaeger:</span></span><br><span class="line"><span class="hljs-attr">    http-sender:</span></span><br><span class="line"><span class="hljs-attr">      url:</span> <span class="hljs-attr">http://xxxxxx/traces</span> <span class="hljs-comment">#采集端接受数据的接口地址</span></span><br><span class="line"><span class="hljs-attr">  spring:</span></span><br><span class="line"><span class="hljs-attr">    web:</span></span><br><span class="line"><span class="hljs-attr">      skip-pattern:</span> <span class="hljs-string">"/prometheus|/health"</span> <span class="hljs-comment"># url过滤名单</span></span><br><span class="line"><span class="hljs-attr">      enabled:</span> <span class="hljs-literal">true</span></span><br><span class="line"><span class="hljs-attr">      client:</span></span><br><span class="line"><span class="hljs-attr">        enabled:</span> <span class="hljs-literal">true</span></span><br></pre></td></tr></table></figure>

<p>此时便已经接入了追踪系统，并可以通过界面进行相应请求接口的查询。</p>
<ol start="2">
<li>接入jdbc的监控也是和web一样的简单。</li>
</ol>
<p>只需要maven依赖即可</p>
<figure class="highlight xml hljs"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="hljs-tag">&lt;<span class="hljs-name">dependency</span>&gt;</span></span><br><span class="line">	<span class="hljs-tag">&lt;<span class="hljs-name">groupId</span>&gt;</span>io.opentracing.contrib<span class="hljs-tag">&lt;/<span class="hljs-name">groupId</span>&gt;</span></span><br><span class="line">	<span class="hljs-tag">&lt;<span class="hljs-name">artifactId</span>&gt;</span>opentracing-spring-cloud-jdbc-starter<span class="hljs-tag">&lt;/<span class="hljs-name">artifactId</span>&gt;</span></span><br><span class="line">	<span class="hljs-tag">&lt;<span class="hljs-name">version</span>&gt;</span>0.2.2<span class="hljs-tag">&lt;/<span class="hljs-name">version</span>&gt;</span></span><br><span class="line"><span class="hljs-tag">&lt;/<span class="hljs-name">dependency</span>&gt;</span></span><br></pre></td></tr></table></figure>

<p>3.Redis<br>在微诉的用户中心中，使用redis进行用户状态的存储，因此也添加相应的依赖进行redis操作的监控。</p>
<p>添加maven依赖</p>
<figure class="highlight xml hljs"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="hljs-tag">&lt;<span class="hljs-name">dependency</span>&gt;</span></span><br><span class="line">      <span class="hljs-tag">&lt;<span class="hljs-name">groupId</span>&gt;</span>io.opentracing.contrib<span class="hljs-tag">&lt;/<span class="hljs-name">groupId</span>&gt;</span></span><br><span class="line">      <span class="hljs-tag">&lt;<span class="hljs-name">artifactId</span>&gt;</span>opentracing-redis-jedis<span class="hljs-tag">&lt;/<span class="hljs-name">artifactId</span>&gt;</span></span><br><span class="line">      <span class="hljs-tag">&lt;<span class="hljs-name">version</span>&gt;</span>0.0.15<span class="hljs-tag">&lt;/<span class="hljs-name">version</span>&gt;</span></span><br><span class="line">      <span class="hljs-tag">&lt;<span class="hljs-name">scope</span>&gt;</span>compile<span class="hljs-tag">&lt;/<span class="hljs-name">scope</span>&gt;</span></span><br><span class="line"><span class="hljs-tag">&lt;/<span class="hljs-name">dependency</span>&gt;</span></span><br></pre></td></tr></table></figure>

<p>因为使用的springboot版本是1.x，opentracing组织尚未提供该版本下的spring对应的类库。如果需要使用，需要使用对应的jedis或者自己重写RedisTemplate。在这里演示的是重写RedisTemplate的方式。在springboot2.x版本中，更为简单只需要引入对应的类库，并注入RedisConnectionFactory的时候进行埋点即可。</p>
<p>4、其他类库比如RestTemplate、Feigh、RabbitMq的接入方式也是类似的套路，引入类库，简单的已经可以使用，如果是rabbitmq需要和redis一样，对工厂bean进行简单埋点配置操作即可。</p>
<h2 id="配置项目"><a href="#配置项目" class="headerlink" title="配置项目"></a>配置项目</h2><ul>
<li><p>URL过滤配置</p>
<ol>
<li>跳过监控的配置项：opentracing.spring.web.skip-pattern = string，添加自定义的规则后，默认的规则会被覆盖。</li>
<li>默认的URL表达式：<blockquote>
<p>/api-docs.<em>|/autoconfig|/configprops|/dump|/health|/info|/metrics.</em>|
/mappings|/swagger.<em>|.</em>\.png|.<em>\.css|.</em>\.js|.*\.html|/favicon.ico|/hystrix.stream</p>
</blockquote>
</li>
</ol>
</li>
<li><p>采样率设置</p>
<blockquote>
<p> 支持三种采样方式：全量采样、概率采样、速率采样</p>
<ol>
<li>全量采样，配置方式 opentracing.jaeger.const-sampler.decision = true | false</li>
<li>概率采样，对于每次请求是否被采样的一个概率值（0.0~1.0之间），opentracing.jaeger.probabilistic-sampler.sampling-rate = value</li>
<li>速率采样，每秒采样的最大数量，当值处于0-1之间就是全采样，opentracing.jaeger.rate-limiting-max-traces-per-second = value</li>
</ol>
</blockquote>
</li>
<li><p>关闭追踪相关</p>
<ol>
<li>opentracing.spring.web.client.enabled = true | false , 是否开启RestTemplate的监控，即RestTemplate调用外部接口。</li>
<li>opentracing.spring.web.enabled = true | false ，是否开启filter的监控，即项目自身暴露的web接口</li>
<li>opentracing.jaeger.enabled = true | false , 是否开启Jaeger功能。</li>
<li>opentracing.jaeger.log-spans = true | false , 是否发送spans时输出日志，生产环境建议关闭该项，调试时可以开启</li>
</ol>
</li>
</ul>
<h2 id="接入jager后进行接口的性能对比测试。"><a href="#接入jager后进行接口的性能对比测试。" class="headerlink" title="接入jager后进行接口的性能对比测试。"></a>接入jager后进行接口的性能对比测试。</h2><p>使用jmeter对接口进行了性能对比测试，对比结果展示如下。从对比的结果来看，jaeger对系统的性能影响还是比较小的。</p>
<blockquote>
<p>todo 后期补充对比图</p>
</blockquote>
<h2 id="Jaeger接入过程中遇到的问题"><a href="#Jaeger接入过程中遇到的问题" class="headerlink" title="Jaeger接入过程中遇到的问题"></a>Jaeger接入过程中遇到的问题</h2><p>在性能测试的过程中，遇到了如下几个问题待解决：</p>
<ol>
<li>在并发的情况下，采集系统的采集数据会出现丢失的情况。在使用采样率比较低的时候，并不会出现这种情况。</li>
<li>在数据量比较集中的时候，jaeger的ui有bug，不会全部显示完整的采样数据，则会选择其中的一部分进行显示。通过打印日志可以看出其实数据已经上传并保存到了jaeger的存储中了，也可以通过tracer的id进行数据的查询并展示。<br>在跟换了存储方式以及替换了新版的ui之后问题已经修复。</li>
</ol>
<h2 id="相关连接地址"><a href="#相关连接地址" class="headerlink" title="相关连接地址"></a>相关连接地址</h2><p>OpenTracing 中文文档 ： <a href="hhttps://wu-sheng.gitbooks.io/opentracing-io/content/?spm=a2c4e.11153940.blogcont514488.34.11b711f4WUomJG" target="_blank" rel="noopener">https://wu-sheng.gitbooks.io/opentracing-io/content</a></p>
<p>Jaeger官网 ： <a href="https://www.jaegertracing.io" target="_blank" rel="noopener">https://www.jaegertracing.io</a></p>

        </div>
        
        <div class="level is-size-7 is-uppercase">
            <div class="level-start">
                <div class="level-item">
                    <span class="is-size-6 has-text-grey has-mr-7">#</span>
                    <a class="has-link-grey -link" href="/blog/tags/Jaeger/">Jaeger</a>, <a class="has-link-grey -link" href="/blog/tags/Opentracing/">Opentracing</a>, <a class="has-link-grey -link" href="/blog/tags/分布式追踪/">分布式追踪</a>
                </div>
            </div>
        </div>
        
        
        
        <div class="social-share"></div>
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/social-share.js@1.0.16/dist/css/share.min.css">
<script src="https://cdn.jsdelivr.net/npm/social-share.js@1.0.16/dist/js/social-share.min.js"></script>
        
    </div>
</div>





<div class="card card-transparent">
    <div class="level post-navigation is-flex-wrap is-mobile">
        
        <div class="level-start">
            <a class="level level-item has-link-grey  article-nav-prev" href="/blog/2019/06/16/Spinnaker/">
                <i class="level-item fas fa-chevron-left"></i>
                <span class="level-item">Spinnaker</span>
            </a>
        </div>
        
        
        <div class="level-end">
            <a class="level level-item has-link-grey  article-nav-next" href="/blog/2019/06/01/个人简历/">
                <span class="level-item">个人简历</span>
                <i class="level-item fas fa-chevron-right"></i>
            </a>
        </div>
        
    </div>
</div>



<div class="card">
    <div class="card-content">
        <h3 class="title is-5 has-text-weight-normal">评论</h3>
        
<div id="valine-thread" class="content"></div>
<script src="//cdn1.lncld.net/static/js/3.0.4/av-min.js"></script>
<script src="//unpkg.com/valine/dist/Valine.min.js"></script>
<script>
    new Valine({
        el: '#valine-thread' ,
        notify: true,
        verify: true,
        app_id: 'Cfy3y0Tw4oUsMP7lu1bPk87x-gzGzoHsz',
        app_key: 'mLz3X3oCrA0MbYHCANuEtPfD',
        placeholder: 'ヾﾉ≧∀≦)o来啊，快活啊!'
    });
</script>

    </div>
</div>
</div>
                
                




<div class="column is-4-tablet is-4-desktop is-3-widescreen  has-order-3 column-right ">
    
        
<div class="card widget">
    <div class="card-content">
        <nav class="level">
            <div class="level-item has-text-centered">
                <div>
                    
                        <img class="image is-128x128 has-mb-6" src="/blog/images/avatar.JPG" alt="Vincent Wang">
                    
                    
                    <p class="is-size-4 is-block">
                        Vincent Wang
                    </p>
                    
                    
                    <p class="is-size-6 is-block is-flex-center has-text-grey">
                        <i class="fas  fa-briefcase has-mr-7"></i>
                        <span>Engineer</span>
                    </p>
                    
                    
                    <p class="is-size-6 is-flex is-flex-center has-text-grey">
                        <i class="fas fa-map-marker-alt has-mr-7"></i>
                        <span>Beijing</span>
                    </p>
                    
                </div>
            </div>
        </nav>
        <nav class="level is-mobile">
            <div class="level-item has-text-centered is-marginless">
                <div>
                    <p class="heading">
                        文章
                    </p>
                    <p class="title has-text-weight-normal">
                        3
                    </p>
                </div>
            </div>
            <div class="level-item has-text-centered is-marginless">
                <div>
                    <p class="heading">
                        分类
                    </p>
                    <p class="title has-text-weight-normal">
                        6
                    </p>
                </div>
            </div>
            <div class="level-item has-text-centered is-marginless">
                <div>
                    <p class="heading">
                        标签
                    </p>
                    <p class="title has-text-weight-normal">
                        13
                    </p>
                </div>
            </div>
        </nav>
        <div class="level">
            <a class="level-item button is-link is-rounded" href="https://github.com/sunnywa" target="_blank">
                关注我</a>
        </div>
        
        
        <div class="level is-mobile">
            
            <a class="level-item button is-white is-marginless" target="_blank" title="Github" href="https://github.com/sunnywa">
                
                <i class="fab fa-github"></i>
                
            </a>
            
            <a class="level-item button is-white is-marginless" target="_blank" title="Facebook" href="https://facebook.com">
                
                <i class="fab fa-facebook"></i>
                
            </a>
            
            <a class="level-item button is-white is-marginless" target="_blank" title="Twitter" href="https://twitter.com">
                
                <i class="fab fa-twitter"></i>
                
            </a>
            
            <a class="level-item button is-white is-marginless" target="_blank" title="Dribbble" href="https://dribbble.com">
                
                <i class="fab fa-dribbble"></i>
                
            </a>
            
            <a class="level-item button is-white is-marginless" target="_blank" title="RSS" href="/blog/">
                
                <i class="fas fa-rss"></i>
                
            </a>
            
        </div>
        
    </div>
</div>
    
        
    
        

<div class="card widget">
    <div class="card-content">
        <div class="menu">
        <h3 class="menu-label">
            链接
        </h3>
        <ul class="menu-list">
        
            <li>
                <a class="level is-mobile" href="https://hexo.io" target="_blank">
                    <span class="level-left">
                        <span class="level-item">Hexo</span>
                    </span>
                    <span class="level-right">
                        <span class="level-item tag">hexo.io</span>
                    </span>
                </a>
            </li>
        
            <li>
                <a class="level is-mobile" href="https://github.com/sunnywa" target="_blank">
                    <span class="level-left">
                        <span class="level-item">Vincent</span>
                    </span>
                    <span class="level-right">
                        <span class="level-item tag">github.com</span>
                    </span>
                </a>
            </li>
        
        </ul>
        </div>
    </div>
</div>


    
        
<div class="card widget">
    <div class="card-content">
        <div class="menu">
            <h3 class="menu-label">
                分类
            </h3>
            <ul class="menu-list">
            <li>
        <a class="level is-marginless" href="/blog/categories/APM/">
            <span class="level-start">
                <span class="level-item">APM</span>
            </span>
            <span class="level-end">
                <span class="level-item tag">1</span>
            </span>
        </a><ul><li>
        <a class="level is-marginless" href="/blog/categories/APM/Opentracing/">
            <span class="level-start">
                <span class="level-item">Opentracing</span>
            </span>
            <span class="level-end">
                <span class="level-item tag">1</span>
            </span>
        </a></li></ul></li><li>
        <a class="level is-marginless" href="/blog/categories/CNCF/">
            <span class="level-start">
                <span class="level-item">CNCF</span>
            </span>
            <span class="level-end">
                <span class="level-item tag">1</span>
            </span>
        </a><ul><li>
        <a class="level is-marginless" href="/blog/categories/CNCF/Spinnaker/">
            <span class="level-start">
                <span class="level-item">Spinnaker</span>
            </span>
            <span class="level-end">
                <span class="level-item tag">1</span>
            </span>
        </a></li></ul></li><li>
        <a class="level is-marginless" href="/blog/categories/个人/">
            <span class="level-start">
                <span class="level-item">个人</span>
            </span>
            <span class="level-end">
                <span class="level-item tag">1</span>
            </span>
        </a><ul><li>
        <a class="level is-marginless" href="/blog/categories/个人/简历/">
            <span class="level-start">
                <span class="level-item">简历</span>
            </span>
            <span class="level-end">
                <span class="level-item tag">1</span>
            </span>
        </a></li></ul></li>
            </ul>
        </div>
    </div>
</div>
    
        
<div class="card widget">
    <div class="card-content">
        <h3 class="menu-label">
            标签云
        </h3>
        <a href="/blog/tags/CD/" style="font-size: 10px;">CD</a> <a href="/blog/tags/Jaeger/" style="font-size: 10px;">Jaeger</a> <a href="/blog/tags/Opentracing/" style="font-size: 10px;">Opentracing</a> <a href="/blog/tags/Spinnaker/" style="font-size: 10px;">Spinnaker</a> <a href="/blog/tags/devops/" style="font-size: 10px;">devops</a> <a href="/blog/tags/pmp/" style="font-size: 10px;">pmp</a> <a href="/blog/tags/云计算/" style="font-size: 10px;">云计算</a> <a href="/blog/tags/分布式追踪/" style="font-size: 10px;">分布式追踪</a> <a href="/blog/tags/后端研发/" style="font-size: 10px;">后端研发</a> <a href="/blog/tags/容器开发/" style="font-size: 10px;">容器开发</a> <a href="/blog/tags/微服务/" style="font-size: 10px;">微服务</a> <a href="/blog/tags/持续部署/" style="font-size: 10px;">持续部署</a> <a href="/blog/tags/简历/" style="font-size: 10px;">简历</a>
    </div>
</div>

    
        
<div class="card widget">
    <div class="card-content">
        <h3 class="menu-label">
            最新文章
        </h3>
        
        <article class="media">
            
            <a href="/blog/2019/06/16/Spinnaker/" class="media-left">
                <p class="image is-64x64">
                    <img class="thumbnail" src="/blog/gallery/thumbnails/640.jpeg" alt="Spinnaker">
                </p>
            </a>
            
            <div class="media-content">
                <div class="content">
                    <div><time class="has-text-grey is-size-7 is-uppercase" datetime="2019-06-16T05:52:09.000Z">2019-06-16</time></div>
                    <a href="/blog/2019/06/16/Spinnaker/" class="has-link-black-ter is-size-6">Spinnaker</a>
                    <p class="is-size-7 is-uppercase">
                        <a class="has-link-grey -link" href="/blog/categories/CNCF/">CNCF</a> / <a class="has-link-grey -link" href="/blog/categories/CNCF/Spinnaker/">Spinnaker</a>
                    </p>
                </div>
            </div>
        </article>
        
        <article class="media">
            
            <a href="/blog/2019/06/02/Opentracing-Jaeger/" class="media-left">
                <p class="image is-64x64">
                    <img class="thumbnail" src="/blog/gallery/thumbnails/opentracing.png" alt="Opentracing &amp; Jaeger">
                </p>
            </a>
            
            <div class="media-content">
                <div class="content">
                    <div><time class="has-text-grey is-size-7 is-uppercase" datetime="2019-06-02T09:10:52.000Z">2019-06-02</time></div>
                    <a href="/blog/2019/06/02/Opentracing-Jaeger/" class="has-link-black-ter is-size-6">Opentracing &amp; Jaeger</a>
                    <p class="is-size-7 is-uppercase">
                        <a class="has-link-grey -link" href="/blog/categories/APM/">APM</a> / <a class="has-link-grey -link" href="/blog/categories/APM/Opentracing/">Opentracing</a>
                    </p>
                </div>
            </div>
        </article>
        
        <article class="media">
            
            <a href="/blog/2019/06/01/个人简历/" class="media-left">
                <p class="image is-64x64">
                    <img class="thumbnail" src="/blog/gallery/thumbnails/deer.jpg" alt="个人简历">
                </p>
            </a>
            
            <div class="media-content">
                <div class="content">
                    <div><time class="has-text-grey is-size-7 is-uppercase" datetime="2019-06-01T15:27:02.000Z">2019-06-01</time></div>
                    <a href="/blog/2019/06/01/个人简历/" class="has-link-black-ter is-size-6">个人简历</a>
                    <p class="is-size-7 is-uppercase">
                        <a class="has-link-grey -link" href="/blog/categories/个人/">个人</a> / <a class="has-link-grey -link" href="/blog/categories/个人/简历/">简历</a>
                    </p>
                </div>
            </div>
        </article>
        
    </div>
</div>

    
        <div class="card widget">
    <div class="card-content">
        <div class="menu">
        <h3 class="menu-label">
            归档
        </h3>
        <ul class="menu-list">
        
        <li>
            <a class="level is-marginless" href="/blog/archives/2019/06/">
                <span class="level-start">
                    <span class="level-item">六月 2019</span>
                </span>
                <span class="level-end">
                    <span class="level-item tag">3</span>
                </span>
            </a>
        </li>
        
        </ul>
        </div>
    </div>
</div>
    
        <div class="card widget">
    <div class="card-content">
        <div class="menu">
            <h3 class="menu-label">
                标签
            </h3>
            <div class="field is-grouped is-grouped-multiline">
                
                <div class="control">
                    <a class="tags has-addons" href="/blog/tags/CD/">
                        <span class="tag">CD</span>
                        <span class="tag is-grey">1</span>
                    </a>
                </div>
                
                <div class="control">
                    <a class="tags has-addons" href="/blog/tags/Jaeger/">
                        <span class="tag">Jaeger</span>
                        <span class="tag is-grey">1</span>
                    </a>
                </div>
                
                <div class="control">
                    <a class="tags has-addons" href="/blog/tags/Opentracing/">
                        <span class="tag">Opentracing</span>
                        <span class="tag is-grey">1</span>
                    </a>
                </div>
                
                <div class="control">
                    <a class="tags has-addons" href="/blog/tags/Spinnaker/">
                        <span class="tag">Spinnaker</span>
                        <span class="tag is-grey">1</span>
                    </a>
                </div>
                
                <div class="control">
                    <a class="tags has-addons" href="/blog/tags/devops/">
                        <span class="tag">devops</span>
                        <span class="tag is-grey">1</span>
                    </a>
                </div>
                
                <div class="control">
                    <a class="tags has-addons" href="/blog/tags/pmp/">
                        <span class="tag">pmp</span>
                        <span class="tag is-grey">1</span>
                    </a>
                </div>
                
                <div class="control">
                    <a class="tags has-addons" href="/blog/tags/云计算/">
                        <span class="tag">云计算</span>
                        <span class="tag is-grey">1</span>
                    </a>
                </div>
                
                <div class="control">
                    <a class="tags has-addons" href="/blog/tags/分布式追踪/">
                        <span class="tag">分布式追踪</span>
                        <span class="tag is-grey">1</span>
                    </a>
                </div>
                
                <div class="control">
                    <a class="tags has-addons" href="/blog/tags/后端研发/">
                        <span class="tag">后端研发</span>
                        <span class="tag is-grey">1</span>
                    </a>
                </div>
                
                <div class="control">
                    <a class="tags has-addons" href="/blog/tags/容器开发/">
                        <span class="tag">容器开发</span>
                        <span class="tag is-grey">1</span>
                    </a>
                </div>
                
                <div class="control">
                    <a class="tags has-addons" href="/blog/tags/微服务/">
                        <span class="tag">微服务</span>
                        <span class="tag is-grey">1</span>
                    </a>
                </div>
                
                <div class="control">
                    <a class="tags has-addons" href="/blog/tags/持续部署/">
                        <span class="tag">持续部署</span>
                        <span class="tag is-grey">1</span>
                    </a>
                </div>
                
                <div class="control">
                    <a class="tags has-addons" href="/blog/tags/简历/">
                        <span class="tag">简历</span>
                        <span class="tag is-grey">1</span>
                    </a>
                </div>
                
            </div>
        </div>
    </div>
</div>
    
    
</div>

            </div>
        </div>
    </section>
    <footer class="footer">
    <div class="container">
        <div class="level">
            <div class="level-start has-text-centered-mobile">
                <a class="footer-logo is-block has-mb-6" href="/blog/">
                
                    <img src="/blog/images/logo.png" alt="Opentracing &amp; Jaeger" height="28">
                
                </a>
                <p class="is-size-7">
                &copy; 2019 Vincent Wang&nbsp;
                Powered by <a href="https://hexo.io/" target="_blank">Hexo</a> & <a href="https://github.com/ppoffice/hexo-theme-icarus" target="_blank">Icarus</a>
                
                </p>
            </div>
            <div class="level-end">
            
                <div class="field has-addons is-flex-center-mobile has-mt-5-mobile is-flex-wrap is-flex-middle">
                
                
                <p class="control">
                    <a class="button is-white is-large" target="_blank" title="Creative Commons" href="https://creativecommons.org/">
                        
                        <i class="fab fa-creative-commons"></i>
                        
                    </a>
                </p>
                
                <p class="control">
                    <a class="button is-white is-large" target="_blank" title="Attribution 4.0 International" href="https://creativecommons.org/licenses/by/4.0/">
                        
                        <i class="fab fa-creative-commons-by"></i>
                        
                    </a>
                </p>
                
                <p class="control">
                    <a class="button is-white is-large" target="_blank" title="Download on GitHub" href="https://github.com/ppoffice/hexo-theme-icarus">
                        
                        <i class="fab fa-github"></i>
                        
                    </a>
                </p>
                
                </div>
            
            </div>
        </div>
    </div>
</footer>
    <script src="https://cdn.jsdelivr.net/npm/jquery@3.3.1/dist/jquery.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/moment@2.22.2/min/moment-with-locales.min.js"></script>
<script>moment.locale("zh-CN");</script>


    
    
    
    <script src="/blog/js/animation.js"></script>
    

    
    
    
    <script src="https://cdn.jsdelivr.net/npm/lightgallery@1.6.8/dist/js/lightgallery.min.js" defer></script>
    <script src="https://cdn.jsdelivr.net/npm/justifiedGallery@3.7.0/dist/js/jquery.justifiedGallery.min.js" defer></script>
    <script src="/blog/js/gallery.js" defer></script>
    

    
    

<div id="outdated">
    <h6>Your browser is out-of-date!</h6>
    <p>Update your browser to view this website correctly. <a id="btnUpdateBrowser" href="http://outdatedbrowser.com/">Update
            my browser now </a></p>
    <p class="last"><a href="#" id="btnCloseUpdateBrowser" title="Close">&times;</a></p>
</div>
<script src="https://cdn.jsdelivr.net/npm/outdatedbrowser@1.1.5/outdatedbrowser/outdatedbrowser.min.js" defer></script>
<script>
    document.addEventListener("DOMContentLoaded", function () {
        outdatedBrowser({
            bgColor: '#f25648',
            color: '#ffffff',
            lowerThan: 'flex'
        });
    });
</script>


    
    
<script src="https://cdn.jsdelivr.net/npm/mathjax@2.7.5/unpacked/MathJax.js?config=TeX-MML-AM_CHTML" defer></script>
<script>
document.addEventListener('DOMContentLoaded', function () {
    MathJax.Hub.Config({
        'HTML-CSS': {
            matchFontHeight: false
        },
        SVG: {
            matchFontHeight: false
        },
        CommonHTML: {
            matchFontHeight: false
        },
        tex2jax: {
            inlineMath: [
                ['$','$'],
                ['\\(','\\)']
            ]
        }
    });
});
</script>

    
    

<a id="back-to-top" title="回到顶端" href="javascript:;">
    <i class="fas fa-chevron-up"></i>
</a>
<script src="/blog/js/back-to-top.js" defer></script>


    
    

    
    
    
    

    
    
    
    
    
    <script src="https://cdn.jsdelivr.net/npm/clipboard@2.0.4/dist/clipboard.min.js" defer></script>
    <script src="/blog/js/clipboard.js" defer></script>
    

    
    
    


<script src="/blog/js/main.js" defer></script>

    
    <div class="searchbox ins-search">
    <div class="searchbox-container ins-search-container">
        <div class="searchbox-input-wrapper">
            <input type="text" class="searchbox-input ins-search-input" placeholder="想要查找什么...">
            <span class="searchbox-close ins-close ins-selectable"><i class="fa fa-times-circle"></i></span>
        </div>
        <div class="searchbox-result-wrapper ins-section-wrapper">
            <div class="ins-section-container"></div>
        </div>
    </div>
</div>
<script>
    (function (window) {
        var INSIGHT_CONFIG = {
            TRANSLATION: {
                POSTS: '文章',
                PAGES: '页面',
                CATEGORIES: '分类',
                TAGS: '标签',
                UNTITLED: '(无标题)',
            },
            CONTENT_URL: '/blog/content.json',
        };
        window.INSIGHT_CONFIG = INSIGHT_CONFIG;
    })(window);
</script>
<script src="/blog/js/insight.js" defer></script>
<link rel="stylesheet" href="/blog/css/search.css">
<link rel="stylesheet" href="/blog/css/insight.css">
    
</body>
</html>